Focal adhesion kinase (FAK) is a nonreceptor intracellular tyrosine kinase that plays an essential role in cancer cell adhesion, survival, proliferation, and migration through both its enzymatic activities and scaffolding functions. Overexpression of FAK has been found in many human cancer cells from different origins, which promotes tumor progression and influences clinical outcomes in different classes of human tumors. Therefore, FAK has been considered as a promising target for small molecule anticancer drug development. Many FAK inhibitors targeting different domains of FAK with various mechanisms of functions have been reported, including kinase domain inhibitors, FERM domain inhibitors, and FAT domain inhibitors. In addition, FAK-targeting PROTACs, which can induce the degradation of FAK, have also been developed. In this Perspective, we summarized the progress in the development of small molecular FAK inhibitors and proposed the perspectives for the future development of agents targeting FAK.